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ABSTRACT 

This paper provides status on an applications investigation of the potential for using an expert 
system shell for classification and retrieval of high-resolution digital, color space shuttle closeout 
photography. This NASA funded activity has focused on the use of integrated information 
technologies to intelligently classify and retrieve still imagery from a large, electronically stored 
collection. In this paper a space shuttle processing problem is identified, a working prototype system 
is described, and commercial applications are identified. A conclusion reached is that the developed 
system has distinct advantages over the present manual system and cost efficiencies will result as the 
system is implemented. Further, commercial potential exists for this integrated technology. 

INTRODUCTION 

Research under way at the University of Central Florida (UCF) College of Engineering’s 
Intelligent Multimedia Applications Laboratory (IMAL) is directed at the investigation of the 
feasibility and integration of knowledge-based (expert) systems technology to facilitate the use of a 
large microcomputer-based collection of high-resolution digital photographic images. Specifically, a 
NASA-sponsored research project focuses on the collection, compression, classification, storage, 
retrieval, and transmission of high-resolution Kennedy Space Center (KSC) space shuttle ground 
processing still color photographs ([1],[2],[3]). The following briefly describes this application. 

A pplication Background 

For each space shuttle ground processing cycle, approximately 35,000 chemically developed 8 x 
10 inch still photographs (including copies) document all significant pre-launch and post-landing 
activities. These pictures are an integral part of NASA’s quality control and reliability program that 
assists space shuttle systems engineers in verifying "go for launch" and the condition of the orbiter 
after landing. 

Photographs of key subsystem elements of space shuttle systems (orbiter, main engines, external 
tank, solid rocket boosters, and ground support equipment) are contained in a permanent collection of 
almost two million pictures. Included in these detailed photographs are pumps, rocket nozzles, 
connectors, cabling, control panels, mechanical assemblies, protective tiles, and valves. Presently, this 
collection is manually catalogued and stored in notebook binders, file cabinets, and storage boxes. 

When a KSC systems engineer, or other user, needs an individual photograph or a group of 
related photographs, a search is performed by expert staff specialists. The present system allows 
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network and network protocols to satisfy the demanding network traffic. The use of XTP and FDDI may provide a 
solution. However, more research need to be done in order to incorporate existing networks to XTP and FDDI. 

Remote Data Acquisition 

Data Acquisition has many applications in industry. Some of these applications such as air tunnel 
analysis require large amount of data acquisition and transmission through computer network. This will inevitably 
create a bottleneck effect on the network. The use of XTP and FDDI provides a potential solution to improve the 
performance of the network traffic and thus facilitate the data flowing. 

CONCLUSION 

After performing the audio and video experiments in our testbed, the results have indicated the use of 
XTP and FDDI on multimedia transmission is feasible. Several potential commercial applications are described. 
As multimedia will become an important industry in our nation, more research in multimedia transmission is 
needed in order to provide an edge over our competitors. 

We will cooperate with Carnegie Mellon University and University of Virginia to conduct more 
experiments in the future to utilize more powerful computers, operating systems, audio, and video equipment. 
These experiments will provide higher performance multimedia transmission, benchmarks, research data, and new 
technology to industries. 
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manual searches by mission number, key space shuttle elements, work authorization numbers, or by 
unique image accession number. Retrieval is difficult or impossible without this information. 
Frequently, the needed photo or set is found only after a time-consuming search. This effort is 
repeated at other NASA and contractor locations around the country. 

Retrieved processing photographs have been used to answer critical configuration processing 
questions on numerous occasions. They have also eliminated the need for expensive space shuttle 
disassembly or inspection in closed or hazardous locations. As should be evident, the cost of 
misplaced or lost shuttle photographs is very high when compared to a study indicating that even a 
misfiled document in an average business costs $125, and each lost document costs $350-$700 [4]. 

Because of the importance of these photographs, NASA-KSC has sponsored research to 
investigate more cost-effective methods of system improvement, including the use of expert systems 
for the critical classification and retrieval phases of the Image Database Management System (IDBMS) 
life-cycle process. 

Knowledge- Assisted System Enhancement 

Theoretically and practically, the use of knowledge-based, or expert, systems has potential to 
positively impact NASA image classification and retrieval. This is important because of the large 
quantity of photographs involved and their difficult to describe technical orientation. In the NASA- 
KSC environment, staff specialists have, over the years, developed heuristics for image classification 
and retrieval tasks not commonly known or documented. For example, experienced classification 
specialists have learned to recognize which objects in technically oriented space shuttle subsystem 
pictures are important and which are not. Retrieval specialists also develop an understanding of the 
logical sequence systems engineers and quality technicians use to location required images. This 
knowledge is not possessed by less experienced, substitute, or newly hired classifiers or retrievers 
who replace those that are sick, transferred, or retired. 

Another researcher shares the view that an expert system, functioning as a decision-aid to help 
inexperienced users, can improve the effectiveness and efficiency of image classification and retrieval 
for this class of users [5]. Importantly, several other researchers ([6], [7], [8]) agree that such on-the- 
job task domain expertise provides an ideal environment for expert systems application success. 

PROTOTYPE DEVELOPMENT 

A prototype that integrates expert system technology with an IDBMS has been developed for 
multi-mission NASA space shuttle color image classification and retrieval tasks. The system consists 
of two modules, one for image classification; the other, for image retrieval. They are named 
NAPSAC for NASA Photograph System to Aid Classification; and, PRAISE, for Photo Retrieval And 
Identification System Expert [3]. 

Knowledge acquisition was accomplished by UCF-1MAL researchers who met with KSC 
classification and retrieval domain experts. Numerous interviews were conducted using widely- 
recognized knowledge acquisition and structuring methods ([8], [9], [10]). Resultant image processing 
paradigms, decision criteria, descriptor attributes, and heuristics were structured, modeled, and 
converted into code using LEVEL5 OBJECT (an object-oriented expert system development tool). 

This expert system shell was chosen because of its capabilities for: 

• Developing rule-based and object forms of knowledge representations. 

• Manipulating and displaying user-friendly bitmapped images. 
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• Creating hypertext and object-oriented interfaces. 

• Calling external programs needed to access relational databases (in this case, dBASE III 
PLUS) and image storage devices. 

• Integrating with IBM microcomputer hardware and Microsoft Windows software. 

Image Classification 

The classification module uses the expert system development shell’s hypertext and 
object-oriented features. For example, during image classification, a cataloger views a hierarchy of 
bitmapped object images of space shuttle elements on various screens as shown in Figure 1. The 
screen at the first level pictures the entire space shuttle configuration. "Hyperregions" (shown as 
dashed boxes) outline major shuttle elements (objects)-the orbiter, external tank, solid rocket 
boosters, and ground support equipment. Screen notes (not shown) provide user instructions. If the 
cataloger uses a mouse selection device to point-and-click on the orbiter hyperregion, the orbiter is 
displayed on a second level screen where the bitmapped orbiter object and its forward fuselage, 
payload bay, and tail section hyperregions are outlined. 

After further point-and-click selection (for instance, to the forward fuselage hyperregion), a 
third level screen displays the forward fuselage with flight deck, middeck, and lower deck 
hyperregions. A fourth level selection (for example the flight deck of the forward fuselage) allows 
final object selection of specific subsystems (for example, hand controller or pilot control panel). 

Only four levels of object hyperregion image displays are implemented in this prototype classifier 
system. The reason being that four levels of classification appears more than adequate, after user 
testing, to accommodate image processing paradigms, decision criteria, descriptor attributes, 
heuristics, and data requirements which were identified during the knowledge acquisition process. No 
required information appears lost with this level of specificity. 

A database record is automatically created as a result of the cataloger’s manual input of selected 
image data and navigation through the bitmapped object images. Image record information includes 
work authorization numbers, vehicle and mission information, photographic image data, and a 
sequential image number which is assigned by the expert system. A photo description area is available 
for special cataloger notes or comments. Aspect and sub-aspect entries are entered automatically as a 
result of point-and-click selection. Image record database information is stored on a hard disk, and the 
images are stored separately in a compressed mode on a mass storage device. 

Image Retrieval 

The retrieval module allows engineers and other casual users to locate and display space shuttle 
processing images that have been previously classified and stored. The image retriever expert system 
works much as the classifier does except that it does not create database records. 

During the image retrieval process, the user is first allowed to directly input selected attribute 
values such as a KSC photo or work authorization number, if known, for immediate image retrieval. 

If this information is unknown, the vehicle name, mission number, and/or photo acquisition 
information are identified. Next, the screen sequence of bitmapped space shuttle images described for 
the classifier and illustrated in Figure 1 is used (with point-and-click operation). Attribute values (for 
example, orbiter, forward fuselage, flight deck) are automatically constructed by the expert system 
shell to serve as search query criteria for image retrieval. Like the classifier, four screen displays 
levels in the prototype system were found by KSC users to provide needed retrieval capabilities. 
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Figure 1 . Diagram of classifier/retriever screens with hyperregions indicated. 
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After the query has been structured, the database record is searched using the known attribute 
values. Any "hits" are indicated in a counter screen display for the system user. These records are 
then used to retrieve thumbnail images from the image mass storage device for display. These 
surrogates allow the user to determine image relevancy. 

Using the mouse, a user selects a desired thumbnail image. The system automatically uses the 
image number pointer in the database record to locate the desired image from data storage. Image 
object attribute values (e.g., KSC photo number, mission number, date, etc.) from the database 
record are first displayed. Then, the full screen, full color image is shown after decompression. This 
technique follows die cognitive model used by domain experts as well as that described by other 
researchers [11]. After viewing the image and its associated data, a user can choose to return to the 
screen of thumbnail images for additional selections or may choose to exit the session. Help text 
screens and instructional notes are available throughout the system. 

The retriever and classifier prototype systems enable faster retrieval and classification of still 
photographs (when compared to the old system). Both eliminate or reduce many procedural steps, and 
the point-and-click feature greatly reduces many opportunities for human keyboard error. The expert 
system modules also enforce a standard terminology and single database repository, for the 
classification and retrieval of KSC image records. These improvements have the potential to eliminate 
or greatly reduce temporal classification variations, ensure consistency in classifier and uses 
orientation, and provide a more user-friendly environment. 

PC-LAN VERSION 

The prototype classification and retrieval modules work well for a one-user, single location 
system. However, space shuttle ground processing activities are spread over twenty-five square miles 
in at least six different KSC facilities. In addition, during pre-launch preparation and post-landing 
operations, engineers at locations throughout the country have responsibility for evaluating various 
space shuttle elements. A networked IDBMS is required to support the needs of these users. 

A generalized PC-based local area network (LAN) prototype IDBMS design is shown in Figure 
2. It is consistent with another researcher’s assertion that there are advantages to an open architecture 
which enables seamless image and data access through the use of powerful servers and related desktop 
technology [5], This architecture is organized into four primary modules: production station, file 
server, local user workstations, and remote workstations. The first three modules are connected by a 
high speed (16 megabits per second), token ring LAN network running under Novell Netware 
control. The remote workstations (the fourth module) will eventually be linked to the file server via 
high capacity communication links. 

The purpose of the IDBMS production station is to input photographic images, update the 
record database using the classifier expert system, and compress images. Images can be captured by 
using either a high-resolution photographic color scanner, a television camera, or analog or digital 
still video electronic cameras. 

The knowledge-assisted classifier is used to create a database record of the captured image 
during the acquisition process. Digital images are entered into the system directly from the 
photographic flatbed scanner or a digital still video system. An alternate method is to input analog 
images using a television camera (for 8 x 10 inch photographs) or an analog still video camera. For 
analog image digitization, a commercial analog-to-digital board in the production station 
microcomputer is used to create individual digital image files. Each of these files is then compressed 
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Figure 2. PC-LAN IDBMS. 
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by a JPEG-compatible board also located in the microcomputer. Compression ratios range from 30: 1 
to 35:1, depending on image complexity. 

After image capture, analog-to-digital conversion (if required), and compression at the 
production station, image files are transmitted to the mass storage device attached to the file server. A 
one gigabyte erasable optical disk drive is used for mass image data storage for this prototype system. 
A higher capacity image storage device, in the terabyte range, would be required for an operational 
system. The database records used to locate images are stored as a database file on the file server’s 
separate internal hard disk. 

Images are retrieved for engineering and quality analysis at user workstations. The retrieval 
expert system, resident at each workstation, is used to build a database query for image retrieval and 
display. A query usually results in multiple hits which are displayed as bitmapped thumbnail pictures 
for user review and further selection. The desired image is then decompressed (using a JPEG- 
compatible board in each workstation) in real-time and displayed on a large high-resolution (1024 x 
768 pixel), true color (24-bit) monitor. The image is printed, if needed, using a digital high-resolution 
color printer. 

The final module in this architecture consists of remote workstations. They function the same as 
local KSC user stations for image retrieval and display, except that they are at various locations 
throughout the United States. During remote operation, engineers at these distant locations would 
access the KSC file server records database using the knowledge-assisted retrieval expert system 
resident on each workstation. In operation, compressed image files, retrieved from the image mass 
data storage unit, would be transmitted via existing high-speed transmission links from KSC to 
various user locations. 


IMPORTANCE AND COMMERCIAL POTENTIAL 

The paradigm shift of using PC -based networked digital image data and knowledge-based 
classification and retrieval instead of hard copy prints and manual image management is anticipated to 
result in a significant improvement in closeout image productivity and cost savings for NASA-KSC 
space shuttle processing operations. It is estimated that cost savings/avoidance of 50% are possible as 
a result of implementation of an interactive digital image management system using knowledge-based 
classification and retrieval. 

Also significant is the commercial potential for use of these low-cost integrated technologies in a 
variety of private and public sector environments and applications. Potential environments include 
education and training, travel and tourism, commercial and residential real estate, medicine and 
dentistry, survey and construction, advertising and sales, and presentation and entertainment. Specific 
media requiring improved image management systems include education and training materials, 
medical X-ray and scanned data, satellite and aircraft imagery, high quality photographic collections, 
advertising and related graphic inventories, visual product images, and executive and corporate 
presentation materials. Important is the fact that developed software technologies are fully compatible 
with many existing private and public sector PC-based client-server computer environments. 
Implementation of visual database technologies will require application system analysis and design, 
domain-specific software development, and systems integration. As a result, potential exists to 
stimulate growth in a variety of small business computer service organizations and in the development 
of generic/industry specific software programs. 
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SUMMARY AND CONCLUSIONS 

This paper has briefly reviewed features important to high-capacity photographic image data 
capture, classification, compression, storage, retrieval, and display. Also described was a NASA-KSC 
space shuttle ground processing prototype IDBMS under development which provides knowledge- 
based assistance for image classification and retrieval. Finally, a design for a networked PC-LAN 
IDBMS was presented. A conclusion reached from reviews of the prototype system is that it has 
distinct advantages over the present manual system and cost efficiencies will result as the system is 
implemented. Further, commercial potential exists for this integrated technology. 
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Introduction 

The increasing sophistication of computers has made 
digital manipulation of photographic images (as well as 
other digitally-recorded artifacts, such as audio and 
video) incredibly easy to perform and, as time goes on, 
increasingly difficult to detect. Today, every picture 
appearing in newspapers and magazines has been 
digitally altered to some degree, with the severity 
varying from the trivial (cleaning up "noise" and 
removing distracting backgrounds) to the point, of 
deception (articles of clothing removed, heads attached 
to other people’s bodies, the complete rearrangement of 
city skylines). As the power, flexibility and ubiquity of 
image-altering computers continues to increase, the 
well-known adage that "the photograph doesn’t lie" will 
continue to become an anachronism. 

A solution to this problem comes from a concept called 
Digital Signatures, which incorporates modern 
cryptographic techniques to authenticate electronic mail 
messages. [1] [2] ("Authenticate" in this case means you 
can be sure that the message has not been altered, and 
that the sender’s identity has not been forged.) The 
technique can serve not only to authenticate images, but 
also to help the photographer retain and enforce 
copyright protection when the concept of "electronic 
original" is no longer meaningful. 

Background on Digital Signatures 

The concept of a digital signature builds upon a recent 
encryption technique called "Public Key Encryption" 
[3]. Older encryption/decryption schemes require that 
both the sender and receiver possess the same secret 
"key": the sender uses the key to transform the text 
message into ciphertext, and the receiver uses the same 
key to perform an inverse transformation on the 
ciphertext, revealing the original text message. If the 
correct key transforms the ciphertext into unreadable 
garbage, it is reasonable to conclude that either the 
wrong key is being used, the message has been altered, 
or the sender has been impersonated by someone 
ignorant of the correct key. The historic drawback to 


this secret key encryption scheme has been in the secure 
distribution of keys; key disclosure must occur out-of- 
band, either transmitted via an expensive alternate path 
or arranged when sender and receiver were proximate. 

Public key encryption techniques differ in that they 
enable the recipient of a message to decrypt it using a 
key that is different from the one used by the sender to 
encrypt it. All public key cryptography is based on the 
principle that it is easy to multiply two large prime 
numbers together, but extremely difficult (taking 
perhaps centuries using today's supercomputers) to work 
backwards and uncover the factors that could have been 
used to generate the resulting number. 

Public Key Encryption employs two different keys: a 
private key, which is held by the more security 
conscious party, and a corresponding public key, which 
need not be kept secret. The public key is generated 
based upon the private key, making the pair unique to 
each other. 

The public key scheme is illustrated in Figure 1 and 
works as follows: to send a secret message that only the 
recipient can read, the recipient would first make his/her 
public key known to the sender through any non-sec ure 
medium, such as a letter, a telephone conversation, or a 
newspaper ad. Anyone wishing to send a secure 
message would encrypt the message using this public 
key and send it to the recipient The recipient, having 
sole possession of the corresponding private key, is the 
only one able to decrypt the message. The need to 
transmit a secret key that both parties must possess 
beforehand has been eliminated. The tradeoff in this 
case is that, although only the recipient can read the 
message, anyone who obtains the public key can send a 
message with anonymity. * 


* The described scenario can also be used as the first step in a 
process of exchanging secret keys to allow for conventional 
secure message transmission, eliminating any of the drawbacks 
of the one-way authenticatability. [1], [4 ] 
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